Dynomotion

Group: DynoMotion Message: 6819 From: himykabibble Date: 2/25/2013
Subject: Handling Toolchange Errors
Tom,

I'm wondering how other handle errors in a toolchanger. I've recently built a toolchanger for my knee mill, and I'm now working on a much improved design for my new bed mill. The changer is controlled by a program that runs in its own thread, in response to an M6. Both changers have numerous sensors, so every action can be confirmed. If any error occurs, I throw up a dialog telling the user what went wrong, and stop execution of the G-code program, if any. The problem is, this gives me an inconsistent result, depending on when/where the error occurred. In order to recover, the user needs to put the system in a consistent state, but it's not always easy to figure out what to do. Should I just manually load the next tool, then restart the program from the line after the M6? Can I just do another M6?

Any idea how others deal with this?

Also, I added a PCComm function to stop the interpreter, by calling the same function I call when the Stop button in my GUI is clicked. But I find while this method works perfectly when called from the GUI, it doesn't always work properly when called by the PCComm handler, even though I am invoking it. I sometimes end up with the toolchange thread continuing to run after it should have exited.

Regards,
Ray L.
Group: DynoMotion Message: 6820 From: Tom Kerekes Date: 2/26/2013
Subject: Re: Handling Toolchange Errors
Hi Ray,

Error handling is often the most difficult part of a project.  I don't have that much to offer but I would think if the M6 failed then any resume should repeat it.  Your tool changer code would need to handle whatever state the system is in including with the tool fully loaded.  The Coodinated Motion Library should automatically return the GCode line of the M6 if the Halt occurred before it completed.

Regarding the Stop button from the PCComm handler here is one thought:  It might be a race condition where the Interpreter is waiting fro the M6 KFLOP Thread to finish (Terminate) before it goes on.  Meanwhile the M6 Thread detects an error and sends a "Stop" command and then immediately terminates.  It might be indeterminate whether the Interpreter detects the Halt request or the Termination of the M6 Thread first.  Try adding a big delay after sending the PCComm request before exiting the Thread.  (I can't think of a simple way to do a handshake).

HTH
Regards
TK  


Group: DynoMotion Message: 6821 From: himykabibble Date: 2/26/2013
Subject: Re: Handling Toolchange Errors
Tom,

I *think* I've got both problems sorted - still testing all the failure modes on the ATC, but it's looking pretty good so far. There are some icky situations that can occur on toolchanger errors, but I've added a mechanism to allow the user to either re-try, or abort the toolchange, depending on what the error is. It appears to me if the user aborts, then manually puts the system into the state the GUI indicates it *should* be in (this may mean manually loading the tool that mis-loaded), then all is well, and re-starting from the M6 will do the right thing. For most errors, clearing the error (usually some kind of jam), and then doing a re-try will work properly. For some it won't. I'm still deciding how deep into that I want to go, as it would mean storing more information in Persist about *exactly* what went wrong.

The Stop problem was a bonehead mistake on my part, and it's now working perfectly. I had a typo in my PCComm code.

Had one problem this AM that had me thinking I was losing my mind - I was writing a Persist var, and it was getting changed on me. I could write it, then display it, and it was correct. If I then immediately displayed it a second time, it was wrong. Turned out I was (stupidly) updating it on a timer in the GUI - some old code that didn't belong there any more. I don't even remember why it was ever there....

Regards,
Ray L.

--- In DynoMotion@yahoogroups.com, Tom Kerekes <tk@...> wrote:
>
> Hi Ray,
>
> Error handling is often the most difficult part of a project.  I don't have that much to offer but I would think if the M6 failed then any resume should repeat it.  Your tool changer code would need to handle whatever state the system is in including with the tool fully loaded.  The Coodinated Motion Library should automatically return the GCode line of the M6 if the Halt occurred before it completed.
>
> Regarding the Stop button from the PCComm handler here is one thought:  It might be a race condition where the Interpreter is waiting fro the M6 KFLOP Thread to finish (Terminate) before it goes on.  Meanwhile the M6 Thread detects an error and sends a "Stop" command and then immediately terminates.  It might be indeterminate whether the Interpreter detects the Halt request or the Termination of the M6 Thread first.  Try adding a big delay after sending the PCComm request before exiting the Thread.  (I can't think of a simple way to do a handshake).
>
> HTH
> Regards
> TK  
>
>
>
>
> ________________________________
> From: himykabibble <jagboy1964@...>
> To: DynoMotion@yahoogroups.com
> Sent: Monday, February 25, 2013 5:17 PM
> Subject: [DynoMotion] Handling Toolchange Errors
>
>
>  
> Tom,
>
> I'm wondering how other handle errors in a toolchanger. I've recently built a toolchanger for my knee mill, and I'm now working on a much improved design for my new bed mill. The changer is controlled by a program that runs in its own thread, in response to an M6. Both changers have numerous sensors, so every action can be confirmed. If any error occurs, I throw up a dialog telling the user what went wrong, and stop execution of the G-code program, if any. The problem is, this gives me an inconsistent result, depending on when/where the error occurred. In order to recover, the user needs to put the system in a consistent state, but it's not always easy to figure out what to do. Should I just manually load the next tool, then restart the program from the line after the M6? Can I just do another M6?
>
> Any idea how others deal with this?
>
> Also, I added a PCComm function to stop the interpreter, by calling the same function I call when the Stop button in my GUI is clicked. But I find while this method works perfectly when called from the GUI, it doesn't always work properly when called by the PCComm handler, even though I am invoking it. I sometimes end up with the toolchange thread continuing to run after it should have exited.
>
> Regards,
> Ray L.
>
Group: DynoMotion Message: 6822 From: himykabibble Date: 2/26/2013
Subject: Re: Handling Toolchange Errors
Well, spent most of the day playing with this, and made considerable improvements in the ATC code. It's now pretty darned bullet-proof - I couldn't create any errors that it didn't correctly identify and report. At the end, I decided the best way to handle errors is also the simplest - If *anything* goes wrong during a toolchange, I set all of my internal variables (currently loaded tool, current carousel position, etc.) as though it HAD succeeded, then throw an error dialog up to the user, telling him what went wrong, and prompting him to manually ensure the new tools is loaded properly. This is the easiest way to ensure everything is left in a consistent state, and is very easy for the user to understand. The other approaches I came up with all required either storing a lot more state information, plus having different recovery steps for different kinds of errors - not pretty. Some things, humans are much better at dealing with.

I also added a minor feature - if a tool is requested that is not currently in the carousel, the user is prompted to load it manually. At the next toolchange, he'll be prompted to manually unload it. This allows me to run programs that require more tools than the carousel holds, or, for example, to use my 4" face mill, which doesn't have a TTS shank, and doesn't fit in the carousel anyway. At some point, I'll implement logical-to-physical mapping of tools to carousel slots, and implement "virtual carousels", so I can run programs that use many more tools than the carousel holds, and I'll just have to occasionally swap out the carousel.

Regards,
Ray L.

--- In DynoMotion@yahoogroups.com, "himykabibble" <jagboy1964@...> wrote:
>
> Tom,
>
> I *think* I've got both problems sorted - still testing all the failure modes on the ATC, but it's looking pretty good so far. There are some icky situations that can occur on toolchanger errors, but I've added a mechanism to allow the user to either re-try, or abort the toolchange, depending on what the error is. It appears to me if the user aborts, then manually puts the system into the state the GUI indicates it *should* be in (this may mean manually loading the tool that mis-loaded), then all is well, and re-starting from the M6 will do the right thing. For most errors, clearing the error (usually some kind of jam), and then doing a re-try will work properly. For some it won't. I'm still deciding how deep into that I want to go, as it would mean storing more information in Persist about *exactly* what went wrong.
>
> The Stop problem was a bonehead mistake on my part, and it's now working perfectly. I had a typo in my PCComm code.
>
> Had one problem this AM that had me thinking I was losing my mind - I was writing a Persist var, and it was getting changed on me. I could write it, then display it, and it was correct. If I then immediately displayed it a second time, it was wrong. Turned out I was (stupidly) updating it on a timer in the GUI - some old code that didn't belong there any more. I don't even remember why it was ever there....
>
> Regards,
> Ray L.
>
> --- In DynoMotion@yahoogroups.com, Tom Kerekes <tk@> wrote:
> >
> > Hi Ray,
> >
> > Error handling is often the most difficult part of a project.  I don't have that much to offer but I would think if the M6 failed then any resume should repeat it.  Your tool changer code would need to handle whatever state the system is in including with the tool fully loaded.  The Coodinated Motion Library should automatically return the GCode line of the M6 if the Halt occurred before it completed.
> >
> > Regarding the Stop button from the PCComm handler here is one thought:  It might be a race condition where the Interpreter is waiting fro the M6 KFLOP Thread to finish (Terminate) before it goes on.  Meanwhile the M6 Thread detects an error and sends a "Stop" command and then immediately terminates.  It might be indeterminate whether the Interpreter detects the Halt request or the Termination of the M6 Thread first.  Try adding a big delay after sending the PCComm request before exiting the Thread.  (I can't think of a simple way to do a handshake).
> >
> > HTH
> > Regards
> > TK  
> >
> >
> >
> >
> > ________________________________
> > From: himykabibble <jagboy1964@>
> > To: DynoMotion@yahoogroups.com
> > Sent: Monday, February 25, 2013 5:17 PM
> > Subject: [DynoMotion] Handling Toolchange Errors
> >
> >
> >  
> > Tom,
> >
> > I'm wondering how other handle errors in a toolchanger. I've recently built a toolchanger for my knee mill, and I'm now working on a much improved design for my new bed mill. The changer is controlled by a program that runs in its own thread, in response to an M6. Both changers have numerous sensors, so every action can be confirmed. If any error occurs, I throw up a dialog telling the user what went wrong, and stop execution of the G-code program, if any. The problem is, this gives me an inconsistent result, depending on when/where the error occurred. In order to recover, the user needs to put the system in a consistent state, but it's not always easy to figure out what to do. Should I just manually load the next tool, then restart the program from the line after the M6? Can I just do another M6?
> >
> > Any idea how others deal with this?
> >
> > Also, I added a PCComm function to stop the interpreter, by calling the same function I call when the Stop button in my GUI is clicked. But I find while this method works perfectly when called from the GUI, it doesn't always work properly when called by the PCComm handler, even though I am invoking it. I sometimes end up with the toolchange thread continuing to run after it should have exited.
> >
> > Regards,
> > Ray L.
> >
>